Combining filter rewrite and skip list to optimize sub aggregation #19573

jainankitk · 2025-10-08T21:37:54Z

Description

This change aims at optimizing the sub aggregation by leveraging multi range traversal for top level aggregation, and skip list for the sub aggregation.

I have copied over the BitSetDocIdStream class in OpenSearch from Lucene for now as it is not public, but should look at eventually getting rid of it.

Related Issues

Related to #17447, #19384

Check List

Functionality includes testing.
API changes companion pull request created, if applicable.
Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

…ing sub aggregation Signed-off-by: Ankit Jain <[email protected]>

Signed-off-by: Ankit Jain <[email protected]>

github-actions · 2025-10-08T21:58:40Z

❌ Gradle check result for 82bc95d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Ankit Jain <[email protected]>

github-actions · 2025-10-09T00:13:37Z

❌ Gradle check result for aff3dc6: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Ankit Jain <[email protected]>

asimmahmood1 · 2025-10-09T17:51:02Z

I think big5 snapshot for 3.4 should have skiplist enabled for @timestamp.

I tried out this change on big5, 20% data, sorted by @timestamp (default is no sort).

[ec2-user@ip-172-31-61-197 ~]$ opensearch-benchmark compare -b 75193634-a102-4dc0-bff7-8e77af36a358 -c bb6304f5-89de-4bcb-a4a9-df2d2d3e5a20

/ __ ____ ___ ____ / / ____ / / / __ ) ____ / / ____ ___ ____ / /
/ / / / __ / _ / __ \ / _ / __ / ___/ ___/ __ \ / __ / _ \/ __ \/ ___/ __ \/ __ / __ `/ / ///
/ // / // / / / / // / / /_/ / / / // / / / / // / / / / / // / / / / / / / / // / / / ,<
_/ ./_// //____/_/_,/_/ _// // //_// //_// /// // //_,// //||
/_/

Comparing baseline
TestExecution ID: 75193634-a102-4dc0-bff7-8e77af36a358 TestExecution timestamp: 2025-10-09 16:55:05
TestProcedure: big5
ProvisionConfigInstance: external
User tags: asimmahm=3.3rc2-big5-sorted-range-auto-date-baseline

with contender
TestExecution ID: bb6304f5-89de-4bcb-a4a9-df2d2d3e5a20
TestExecution timestamp: 2025-10-09 17:30:10
TestProcedure: big5
ProvisionConfigInstance: external
User tags: asimmahm=3.3rc2-big5-sorted-range-auto-date-candidate

Metric	Task	Baseline	Contender	%Diff	Diff	Unit
Min Throughput	range-auto-date-histo	1.27014	1.22964	-3.19%	-0.04051	ops/s
Mean Throughput	range-auto-date-histo	1.27187	1.23706	-2.74%	-0.03481	ops/s
Median Throughput	range-auto-date-histo	1.27189	1.23643	-2.79%	-0.03546	ops/s
Max Throughput	range-auto-date-histo	1.2737	1.24412	-2.32%	-0.02958	ops/s
50th percentile latency	range-auto-date-histo	72441.8	77212.8	+6.59% 🔴	4771	ms
90th percentile latency	range-auto-date-histo	83602.1	88060.7	+5.33% 🔴	4458.56	ms
99th percentile latency	range-auto-date-histo	86171	90920.7	+5.51% 🔴	4749.77	ms
100th percentile latency	range-auto-date-histo	86456.5	91181.3	+5.46% 🔴	4724.81	ms
50th percentile service time	range-auto-date-histo	766.19	766.99	0.10%	0.80005	ms
90th percentile service time	range-auto-date-histo	833.148	871.862	4.65%	38.7137	ms
99th percentile service time	range-auto-date-histo	926.331	989.296	+6.80% 🔴	62.9648	ms
100th percentile service time	range-auto-date-histo	1017.5	1078.37	+5.98% 🔴	60.868	ms
error rate	range-auto-date-histo	0	0	0.00%	0	%
Min Throughput	range-auto-date-histo-with-metrics	0.416395	0.428757	2.97%	0.01236	ops/s
Mean Throughput	range-auto-date-histo-with-metrics	0.416564	0.429218	3.04%	0.01265	ops/s
Median Throughput	range-auto-date-histo-with-metrics	0.416517	0.42925	3.06%	0.01273	ops/s
Max Throughput	range-auto-date-histo-with-metrics	0.416853	0.429624	3.06%	0.01277	ops/s
50th percentile latency	range-auto-date-histo-with-metrics	476658	457890	-3.94%	-18767.8	ms
90th percentile latency	range-auto-date-histo-with-metrics	552051	530256	-3.95%	-21794.9	ms
99th percentile latency	range-auto-date-histo-with-metrics	568873	546436	-3.94%	-22437	ms
100th percentile latency	range-auto-date-histo-with-metrics	570720	548213	-3.94%	-22507.8	ms
50th percentile service time	range-auto-date-histo-with-metrics	2379.4	2297.31	-3.45%	-82.0885	ms
90th percentile service time	range-auto-date-histo-with-metrics	2479.82	2396.16	-3.37%	-83.6536	ms
99th percentile service time	range-auto-date-histo-with-metrics	2617.2	2540.42	-2.93%	-76.7789	ms
100th percentile service time	range-auto-date-histo-with-metrics	2619.64	2587.99	-1.21%	-31.6505	ms
error rate	range-auto-date-histo-with-metrics	0	0	0.00%	0	%

asimmahmood1 · 2025-10-09T18:07:48Z

So AutoDateHistogram is different than DateHistogram, will need to add skiplist logic there as well:

range_auto_date_histo_baseline.html
range_auto_date_histo_canddiate.html
range-auto-date-histo-with-metrics_baseline.html
range-auto-date-histo-with-metrics_candidate.html

asimmahmood1 · 2025-10-09T20:50:37Z

Fixed the issue with my setup, it wasn't hitting the skiplist path.

Now with skiplist its 20-30% worst latency. Async profile attached.

Metric	Task	Baseline	Contender	%Diff	Diff	Unit
Min Throughput	range-auto-date-histo	1.27014	0.932343	-26.60% 🟢	-0.3378	ops/s
Mean Throughput	range-auto-date-histo	1.27187	0.949476	-25.35% 🟢	-0.3224	ops/s
Median Throughput	range-auto-date-histo	1.27189	0.950048	-25.30% 🟢	-0.32184	ops/s
Max Throughput	range-auto-date-histo	1.2737	0.964287	-24.29% 🟢	-0.30941	ops/s
50th percentile latency	range-auto-date-histo	72441.8	138114	+90.66% 🔴	65672.4	ms
90th percentile latency	range-auto-date-histo	83602.1	156196	+86.83% 🔴	72593.6	ms
99th percentile latency	range-auto-date-histo	86171	160340	+86.07% 🔴	74169.2	ms
100th percentile latency	range-auto-date-histo	86456.5	160782	+85.97% 🔴	74325.7	ms
50th percentile service time	range-auto-date-histo	766.19	948.317	+23.77% 🔴	182.128	ms
90th percentile service time	range-auto-date-histo	833.148	1005.98	+20.74% 🔴	172.83	ms
99th percentile service time	range-auto-date-histo	926.331	1120.47	+20.96% 🔴	194.14	ms
100th percentile service time	range-auto-date-histo	1017.5	1262.62	+24.09% 🔴	245.123	ms
error rate	range-auto-date-histo	0	0	0.00%	0	%
Min Throughput	range-auto-date-histo-with-metrics	0.416395	0.316358	-24.02% 🟢	-0.10004	ops/s
Mean Throughput	range-auto-date-histo-with-metrics	0.416564	0.316664	-23.98% 🟢	-0.0999	ops/s
Median Throughput	range-auto-date-histo-with-metrics	0.416517	0.316571	-24.00% 🟢	-0.09995	ops/s
Max Throughput	range-auto-date-histo-with-metrics	0.416853	0.317119	-23.93% 🟢	-0.09973	ops/s
50th percentile latency	range-auto-date-histo-with-metrics	476658	666033	+39.73% 🔴	189376	ms
90th percentile latency	range-auto-date-histo-with-metrics	552051	769985	+39.48% 🔴	217934	ms
99th percentile latency	range-auto-date-histo-with-metrics	568873	793254	+39.44% 🔴	224381	ms
100th percentile latency	range-auto-date-histo-with-metrics	570720	795939	+39.46% 🔴	225218	ms
50th percentile service time	range-auto-date-histo-with-metrics	2379.4	3127.52	+31.44% 🔴	748.119	ms
90th percentile service time	range-auto-date-histo-with-metrics	2479.82	3196.6	+28.90% 🔴	716.78	ms
99th percentile service time	range-auto-date-histo-with-metrics	2617.2	3266.26	+24.80% 🔴	649.055	ms
100th percentile service time	range-auto-date-histo-with-metrics	2619.64	3302.11	+26.05% 🔴	682.465	ms
error rate	range-auto-date-histo-with-metrics	0	0	0.00%	0	%

range_auto_date_histo_baseline.html
range_auto_date_histo_canddiate.html
range-auto-date-histo-with-metrics_baseline.html
range-auto-date-histo-with-metrics_candidate.html

asimmahmood1 · 2025-10-09T21:20:10Z

Candidate - slower

One big difference is I see is how much time is spent in org/opensearch/search/aggregations/bucket/terms/LongKeyedBucketOrds$FromMany.add in candidate: 23% vs 3%

Baseline

Signed-off-by: Ankit Jain <[email protected]>

jainankitk · 2025-10-10T05:12:29Z

Thanks @asimmahmood1 for sharing these flame graphs. I am kind of surprised to not see BitSetDocIdStream.count in the flamegraph, especially for range_auto_date_histo_canddiate.html as that allows to efficiently skip over the dense run of docIds. I am wondering if for some reason the below logic is not kicking in as expected:

            if (upToSameBucket) {
                if (sub == NO_OP_COLLECTOR) {
                    // stream.count maybe faster when we don't need to handle sub-aggs
                    long count = stream.count(upToExclusive);

We should also verify the correctness of results by comparing the output.

github-actions · 2025-10-10T06:38:27Z

✅ Gradle check result for 0a9ef40: SUCCESS

asimmahmood1 · 2025-10-10T23:42:02Z

...in/java/org/opensearch/search/aggregations/bucket/histogram/AutoDateHistogramAggregator.java

+            // TODO: SkipListLeafCollector should be used if the getLeafCollector invocation is from
+            // filterRewriteOptimizationContext when parent != null. Removing the check to collect
+            // performance numbers for now
+            return new HistogramSkiplistLeafCollector(singleton, skipper, preparedRounding, getBucketOrds(), sub, this);


So it turns out this preparedRound is too granular, so upToSameBucket is always false,

i.e. within the same skipList block (4k) the min != max.

Auto date histogram does some estimation of rounding, based on some starting interval, so we'll need to same within Skiplist version.

For now the game plan is:

Starting with auto date at the top level, find a reasonable rounding (see getRounding() method)

Then try out for sub agg, where for each top level bucket ord we'll need to cal a new rounding. One which is to use the top level skiplist to find maxDocId and use its value as max.

asimmahmood1 · 2025-10-10T23:43:31Z

Since auto date histo needs additional logic for prepare rounding, if just use data histo as sub agg, it still provides a 10 improvement, from 890ms to 88ms!

curl -XGET 'http://localhost:9200/big5/_search' \
-H 'Content-Type: application/json' \
-d '{
  "size": 0,
  "aggs": {
    "tmax": {
      "range": {
        "field": "metrics.size",
        "ranges": [
          {
            "to": -10
          },
          {
            "from": -10,
            "to": 10
          },
          {
            "from": 10,
            "to": 100
          },
          {
            "from": 100,
            "to": 1000
          },
          {
            "from": 1000,
            "to": 2000
          },
          {
            "from": 2000
          }
        ]
      },
      "aggs": {
        "by_hour": {
          "date_histogram": {
            "field": "@timestamp",
            "calendar_interval": "hour"
          }
        }
      }
    }
  }
}'

Signed-off-by: Ankit Jain <[email protected]>

github-actions · 2025-10-21T20:38:15Z

❌ Gradle check result for 65192fd: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Ankit Jain <[email protected]>

github-actions · 2025-10-21T21:17:44Z

❌ Gradle check result for 866937d: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Signed-off-by: Ankit Jain <[email protected]>

asimmahmood1 · 2025-10-21T22:01:38Z

Let me check if its functionally correct:

curl -XGET 'http://localhost:9200/big5/_search' \
-H 'Content-Type: application/json' \
-d '{
  "size": 0,
  "aggs": {
    "tmax": {
      "range": {
        "field": "metrics.size",
        "ranges": [
          {
            "to": -10
          },
          {
            "from": -10,
            "to": 10
          },
          {
            "from": 10,
            "to": 100
          },
          {
            "from": 100,
            "to": 1000
          },
          {
            "from": 1000,
            "to": 2000
          },
          {
            "from": 2000
          }
        ]
      },
      "aggs": {
        "by_hour": {
          "date_histogram": {
            "field": "@timestamp",
            "calendar_interval": "day"
          }
        }
      }
    }
  }
}'

response

{
  "took": 76,
  "timed_out": false,
  "terminated_early": true,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 10000,
      "relation": "gte"
    },
    "max_score": null,
    "hits": []
  },
  "aggregations": {
    "tmax": {
      "buckets": [
        {
          "key": "*--10.0",
          "to": -10.0,
          "doc_count": 0,
          "by_hour": {
            "buckets": []
          }
        },
        {
          "key": "-10.0-10.0",
          "from": -10.0,
          "to": 10.0,
          "doc_count": 0,
          "by_hour": {
            "buckets": []
          }
        },
        {
          "key": "10.0-100.0",
          "from": 10.0,
          "to": 100.0,
          "doc_count": 0,
          "by_hour": {
            "buckets": []
          }
        },
        {
          "key": "100.0-1000.0",
          "from": 100.0,
          "to": 1000.0,
          "doc_count": 0,
          "by_hour": {
            "buckets": []
          }
        },
        {
          "key": "1000.0-2000.0",
          "from": 1000.0,
          "to": 2000.0,
          "doc_count": 656963,
          "by_hour": {
            "buckets": [
              {
                "key_as_string": "2023-01-01T00:00:00.000Z",
                "key": 1672531200000,
                "doc_count": 161239
              },
              {
                "key_as_string": "2023-01-02T00:00:00.000Z",
                "key": 1672617600000,
                "doc_count": 0
              },
              {
                "key_as_string": "2023-01-03T00:00:00.000Z",
                "key": 1672704000000,
                "doc_count": 0
              },
              {
                "key_as_string": "2023-01-04T00:00:00.000Z",
                "key": 1672790400000,
                "doc_count": 162979
              },
              {
                "key_as_string": "2023-01-05T00:00:00.000Z",
                "key": 1672876800000,
                "doc_count": 0
              },
              {
                "key_as_string": "2023-01-06T00:00:00.000Z",
                "key": 1672963200000,
                "doc_count": 0
              },
              {
                "key_as_string": "2023-01-07T00:00:00.000Z",
                "key": 1673049600000,
                "doc_count": 168685
              },
              {
                "key_as_string": "2023-01-08T00:00:00.000Z",
                "key": 1673136000000,
                "doc_count": 0
              },
              {
                "key_as_string": "2023-01-09T00:00:00.000Z",
                "key": 1673222400000,
                "doc_count": 0
              },
              {
                "key_as_string": "2023-01-10T00:00:00.000Z",
                "key": 1673308800000,
                "doc_count": 164265
              }
            ]
          }
        },
        {
          "key": "2000.0-*",
          "from": 2000.0,
          "doc_count": 503037,
          "by_hour": {
            "buckets": [
              {
                "key_as_string": "2023-01-01T00:00:00.000Z",
                "key": 1672531200000,
                "doc_count": 128761
              },
              {
                "key_as_string": "2023-01-02T00:00:00.000Z",
                "key": 1672617600000,
                "doc_count": 0
              },
              {
                "key_as_string": "2023-01-03T00:00:00.000Z",
                "key": 1672704000000,
                "doc_count": 0
              },
              {
                "key_as_string": "2023-01-04T00:00:00.000Z",
                "key": 1672790400000,
                "doc_count": 127021
              },
              {
                "key_as_string": "2023-01-05T00:00:00.000Z",
                "key": 1672876800000,
                "doc_count": 0
              },
              {
                "key_as_string": "2023-01-06T00:00:00.000Z",
                "key": 1672963200000,
                "doc_count": 0
              },
              {
                "key_as_string": "2023-01-07T00:00:00.000Z",
                "key": 1673049600000,
                "doc_count": 121315
              },
              {
                "key_as_string": "2023-01-08T00:00:00.000Z",
                "key": 1673136000000,
                "doc_count": 0
              },
              {
                "key_as_string": "2023-01-09T00:00:00.000Z",
                "key": 1673222400000,
                "doc_count": 0
              },
              {
                "key_as_string": "2023-01-10T00:00:00.000Z",
                "key": 1673308800000,
                "doc_count": 125735
              }
            ]
          }
        }
      ]
    }
  }
}

github-actions · 2025-10-21T22:19:49Z

❌ Gradle check result for 2d697fc: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

asimmahmood1 · 2025-10-21T22:56:57Z

With metrics

This is with 1% index size so numbersa not comparable to big5's benchmark, plus this is date histogram (with day interval),



curl -XGET 'http://localhost:9200/big5/_search' \
-H 'Content-Type: application/json' \
-d '{
  "size": 0,
  "aggs": {
    "tmax": {
      "range": {
        "field": "metrics.size",
        "ranges": [
          {
            "to": -10
          },
          {
            "from": -10,
            "to": 10
          },
          {
            "from": 10,
            "to": 100
          },
          {
            "from": 100,
            "to": 1000
          },
          {
            "from": 1000,
            "to": 2000
          },
          {
            "from": 2000
          }
        ]
      },
      "aggs": {
        "by_hour": {
          "date_histogram": {
            "field": "@timestamp",
            "calendar_interval": "day"
          },
          "aggs": {
            "tmin": {
              "min": {
                "field": "metrics.tmin"
              }
            },
            "tavg": {
              "avg": {
                "field": "metrics.size"
              }
            },
            "tmax": {
              "max": {
                "field": "metrics.size"
              }
            }
          }
        }
      }
    }
  }
}'

{
  "took": 152,
  "timed_out": false,
  "terminated_early": true,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 10000,
      "relation": "gte"
    },
    "max_score": null,
    "hits": []
  },
  "aggregations": {
    "tmax": {
      "buckets": [
        {
          "key": "*--10.0",
          "to": -10.0,
          "doc_count": 0,
          "by_hour": {
            "buckets": []
          }
        },
        {
          "key": "-10.0-10.0",
          "from": -10.0,
          "to": 10.0,
          "doc_count": 0,
          "by_hour": {
            "buckets": []
          }
        },
        {
          "key": "10.0-100.0",
          "from": 10.0,
          "to": 100.0,
          "doc_count": 0,
          "by_hour": {
            "buckets": []
          }
        },
        {
          "key": "100.0-1000.0",
          "from": 100.0,
          "to": 1000.0,
          "doc_count": 0,
          "by_hour": {
            "buckets": []
          }
        },
        {
          "key": "1000.0-2000.0",
          "from": 1000.0,
          "to": 2000.0,
          "doc_count": 656963,
          "by_hour": {
            "buckets": [
              {
                "key_as_string": "2023-01-01T00:00:00.000Z",
                "key": 1672531200000,
                "doc_count": 161239,
                "tmax": {
                  "value": 1999.0
                },
                "tmin": {
                  "value": 1.0
                },
                "tavg": {
                  "value": 1389.6985220697227
                }
              },
              {
                "key_as_string": "2023-01-02T00:00:00.000Z",
                "key": 1672617600000,
                "doc_count": 0,
                "tmax": {
                  "value": null
                },
                "tavg": {
                  "value": null
                },
                "tmin": {
                  "value": null
                }
              },
              {
                "key_as_string": "2023-01-03T00:00:00.000Z",
                "key": 1672704000000,
                "doc_count": 0,
                "tmax": {
                  "value": null
                },
                "tavg": {
                  "value": null
                },
                "tmin": {
                  "value": null
                }
              },
              {
                "key_as_string": "2023-01-04T00:00:00.000Z",
                "key": 1672790400000,
                "doc_count": 162979,
                "tmax": {
                  "value": 1999.0
                },
                "tmin": {
                  "value": 1.0
                },
                "tavg": {
                  "value": 1395.8868320458464
                }
              },
              {
                "key_as_string": "2023-01-05T00:00:00.000Z",
                "key": 1672876800000,
                "doc_count": 0,
                "tmax": {
                  "value": null
                },
                "tavg": {
                  "value": null
                },
                "tmin": {
                  "value": null
                }
              },
              {
                "key_as_string": "2023-01-06T00:00:00.000Z",
                "key": 1672963200000,
                "doc_count": 0,
                "tmax": {
                  "value": null
                },
                "tavg": {
                  "value": null
                },
                "tmin": {
                  "value": null
                }
              },
              {
                "key_as_string": "2023-01-07T00:00:00.000Z",
                "key": 1673049600000,
                "doc_count": 168685,
                "tmax": {
                  "value": 1999.0
                },
                "tmin": {
                  "value": 1.0
                },
                "tavg": {
                  "value": 1386.1427987076504
                }
              },
              {
                "key_as_string": "2023-01-08T00:00:00.000Z",
                "key": 1673136000000,
                "doc_count": 0,
                "tmax": {
                  "value": null
                },
                "tavg": {
                  "value": null
                },
                "tmin": {
                  "value": null
                }
              },
              {
                "key_as_string": "2023-01-09T00:00:00.000Z",
                "key": 1673222400000,
                "doc_count": 0,
                "tmax": {
                  "value": null
                },
                "tavg": {
                  "value": null
                },
                "tmin": {
                  "value": null
                }
              },
              {
                "key_as_string": "2023-01-10T00:00:00.000Z",
                "key": 1673308800000,
                "doc_count": 164265,
                "tmax": {
                  "value": 7680.0
                },
                "tmin": {
                  "value": 1.0
                },
                "tavg": {
                  "value": 1405.8943536358931
                }
              }
            ]
          }
        },
        {
          "key": "2000.0-*",
          "from": 2000.0,
          "doc_count": 503037,
          "by_hour": {
            "buckets": [
              {
                "key_as_string": "2023-01-01T00:00:00.000Z",
                "key": 1672531200000,
                "doc_count": 128761,
                "tmax": {
                  "value": 9999.0
                },
                "tmin": {
                  "value": 1.0
                },
                "tavg": {
                  "value": 4080.2221169453483
                }
              },
              {
                "key_as_string": "2023-01-02T00:00:00.000Z",
                "key": 1672617600000,
                "doc_count": 0,
                "tmax": {
                  "value": null
                },
                "tavg": {
                  "value": null
                },
                "tmin": {
                  "value": null
                }
              },
              {
                "key_as_string": "2023-01-03T00:00:00.000Z",
                "key": 1672704000000,
                "doc_count": 0,
                "tmax": {
                  "value": null
                },
                "tavg": {
                  "value": null
                },
                "tmin": {
                  "value": null
                }
              },
              {
                "key_as_string": "2023-01-04T00:00:00.000Z",
                "key": 1672790400000,
                "doc_count": 127021,
                "tmax": {
                  "value": 9999.0
                },
                "tmin": {
                  "value": 1.0
                },
                "tavg": {
                  "value": 3936.182402909755
                }
              },
              {
                "key_as_string": "2023-01-05T00:00:00.000Z",
                "key": 1672876800000,
                "doc_count": 0,
                "tmax": {
                  "value": null
                },
                "tavg": {
                  "value": null
                },
                "tmin": {
                  "value": null
                }
              },
              {
                "key_as_string": "2023-01-06T00:00:00.000Z",
                "key": 1672963200000,
                "doc_count": 0,
                "tmax": {
                  "value": null
                },
                "tavg": {
                  "value": null
                },
                "tmin": {
                  "value": null
                }
              },
              {
                "key_as_string": "2023-01-07T00:00:00.000Z",
                "key": 1673049600000,
                "doc_count": 121315,
                "tmax": {
                  "value": 9999.0
                },
                "tmin": {
                  "value": 1.0
                },
                "tavg": {
                  "value": 4009.97747187075
                }
              },
              {
                "key_as_string": "2023-01-08T00:00:00.000Z",
                "key": 1673136000000,
                "doc_count": 0,
                "tmax": {
                  "value": null
                },
                "tavg": {
                  "value": null
                },
                "tmin": {
                  "value": null
                }
              },
              {
                "key_as_string": "2023-01-09T00:00:00.000Z",
                "key": 1673222400000,
                "doc_count": 0,
                "tmax": {
                  "value": null
                },
                "tavg": {
                  "value": null
                },
                "tmin": {
                  "value": null
                }
              },
              {
                "key_as_string": "2023-01-10T00:00:00.000Z",
                "key": 1673308800000,
                "doc_count": 125735,
                "tmax": {
                  "value": 9999.0
                },
                "tmin": {
                  "value": 1.0
                },
                "tavg": {
                  "value": 3978.001455442001
                }
              }
            ]
          }
        }
      ]
    }
  }
}

Signed-off-by: Ankit Jain <[email protected]>

github-actions · 2025-10-22T04:59:10Z

❌ Gradle check result for 729bf0a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Combining filter rewrite and skip list approaches for further optimiz…

469cf51

…ing sub aggregation Signed-off-by: Ankit Jain <[email protected]>

jainankitk requested review from a team, Bukhtawar, CEHENKLE, Rishikesh1159, anasalkouz, andrross, ashking94, cwperks, dbwiddis, gbbafna, jed326, kotwanikunal, mch2, msfroh, owaiskazi19, reta, sachinpkale, saratvemulapalli, shwetathareja and sohami as code owners October 8, 2025 21:37

jainankitk marked this pull request as draft October 8, 2025 21:38

Removing parent aggregation check for perf benchmark

e20f702

Signed-off-by: Ankit Jain <[email protected]>

jainankitk changed the title ~~Combining filter rewrite and skip list approaches for further optimiz…~~ Combining filter rewrite and skip list to optimize sub aggregation Oct 8, 2025

Adding changelog entry

82bc95d

Signed-off-by: Ankit Jain <[email protected]>

Applying the skip list optimization for AutoDateHistogram

aff3dc6

Signed-off-by: Ankit Jain <[email protected]>

jainankitk marked this pull request as ready for review October 9, 2025 00:04

Addressing checkstyle failures

1c29540

Signed-off-by: Ankit Jain <[email protected]>

opensearch-ci-bot mentioned this pull request Oct 9, 2025

[AUTOCUT] Gradle Check Flaky Test Report for InstallPluginCommandTests #19256

Open

Minor bug fix

0a9ef40

Signed-off-by: Ankit Jain <[email protected]>

asimmahmood1 reviewed Oct 10, 2025

View reviewed changes

jainankitk requested a review from bowenlan-amzn October 16, 2025 18:46

jainankitk added 6 commits October 21, 2025 11:55

Merge branch 'main' into agg-perf

8d4ccf7

Updating to lucene 10.4 snapshot

3cd5f64

Signed-off-by: Ankit Jain <[email protected]>

Using bulk collect APIs in Lucene

2c657e7

Signed-off-by: Ankit Jain <[email protected]>

Apply spotless

879bfaf

Signed-off-by: Ankit Jain <[email protected]>

Fixing build issues after changing to 10.4 lucene snapshot

128b9b4

Signed-off-by: Ankit Jain <[email protected]>

Fixing Lucene103Codec references in test files

65192fd

Signed-off-by: Ankit Jain <[email protected]>

jainankitk added 2 commits October 21, 2025 13:48

Updating Lucene version for ES

931614c

Signed-off-by: Ankit Jain <[email protected]>

Merge branch 'main' into agg-perf

866937d

Signed-off-by: Ankit Jain <[email protected]>

Updating Lucene codec version for ES

2d697fc

Signed-off-by: Ankit Jain <[email protected]>

jainankitk added 3 commits October 21, 2025 20:49

Adding CompositeCodec104 changes

e04a1b4

Signed-off-by: Ankit Jain <[email protected]>

Remaining CompositeCodec104 changes

45512cd

Signed-off-by: Ankit Jain <[email protected]>

Final CompositeCodec104 changes

729bf0a

Signed-off-by: Ankit Jain <[email protected]>

opensearch-ci-bot mentioned this pull request Oct 22, 2025

[AUTOCUT] Gradle Check Flaky Test Report for FullClusterRestartIT #15813

Open

Combining filter rewrite and skip list to optimize sub aggregation #19573

Are you sure you want to change the base?

Combining filter rewrite and skip list to optimize sub aggregation #19573

Conversation

jainankitk commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issues

Check List

Uh oh!

github-actions bot commented Oct 8, 2025

Uh oh!

github-actions bot commented Oct 9, 2025

Uh oh!

asimmahmood1 commented Oct 9, 2025

Uh oh!

asimmahmood1 commented Oct 9, 2025

Uh oh!

asimmahmood1 commented Oct 9, 2025

Uh oh!

asimmahmood1 commented Oct 9, 2025

Candidate - slower

Baseline

Uh oh!

jainankitk commented Oct 10, 2025

Uh oh!

github-actions bot commented Oct 10, 2025

Uh oh!

asimmahmood1 Oct 10, 2025

Choose a reason for hiding this comment

Uh oh!

asimmahmood1 commented Oct 10, 2025

Uh oh!

github-actions bot commented Oct 21, 2025

Uh oh!

github-actions bot commented Oct 21, 2025

Uh oh!

asimmahmood1 commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 21, 2025

Uh oh!

asimmahmood1 commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

With metrics

Uh oh!

github-actions bot commented Oct 22, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jainankitk commented Oct 8, 2025 •

edited

Loading

asimmahmood1 commented Oct 21, 2025 •

edited

Loading

asimmahmood1 commented Oct 21, 2025 •

edited

Loading